Computing Parallelism in Discourse 



0^ 



> 

o 
o 
vn 
o 
l> 

Oh 



> 

X 
S3 



Claire Gardent 

Computational Linguistics 
Universitat des Saarlandes 

Saarbriicken, Germany 
claire@coli . uni-sb . de 

Abstract 

Although much has been said about paralleHsm 
in discourse, a formal, computational theory 
of parallelism structure is still outstanding. In 
this paper, we present a theory which given two 
parallel utterances predicts which are the par- 
allel elements. The theory consists of a sorted, 
higher-order abductive calculus and we show 
that it reconciles the insights of discourse the- 
ories of parallelism with those of Higher- Order 
Unification approaches to discourse semantics, 
thereby providing a natural framework in which 
to capture the effect of parallelism on discourse 
semantics. 

1 Introduction 

Both Higher-Order Unification (HOU) approaches to 



ai, 1996 ; Gardent et a/., 1996 ; pardent and Kohlhase 



discourse semantics [Dalrymple et ai, 1991 ; Bhieber ei 



1996 and disc ourse theories of parallelism |Hobbs, 199C 



Kehler, 1995| | assume parallelism structuration: given 
a pair of parallel utterances, the parallel elements are 
taken as given. 

This assumption clearly undermines the predictive 
power of a theory. If parallel elements are stipulated 
rather than predicted, conclusions based on parallelism 
remain controversial: what would happen if the parallel 
elements were others? And more crucially, what con- 
straints can we put on parallelism determination (when 
can we rule out a pair as not parallel?) 

In this paper, we present a theory of parallelism which 
goes one step towards answering this objection. Given 
two utterances, the theory predicts which of the ele- 
ments occurring in these utterances are parallel to each 
other. The proposed theory has one additional impor- 
tant advantage: it incorporates HOU as a main compo- 
nent of parallelism theory thereby permitting an integra- 
tion of the HOU approach to discourse semantics with 
discourse theories of parallelism. The resulting frame- 
work permits a natural modelling of the oft en observed 
effect of parallelism on discourse semantics | Lang, 1977 ; 
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Ashcr, 1993|. We show in particular that it correctly 



captures the interaction of VP-ellipsis and gapping with 
parallelism. More generally however, the hope is that it 
also provides an adequate basis for capturing the inter- 
action of parallelism with such discourse phenomena as 
deaccenting, anaphora and quantification. 



Our approach departs from |Grover et al., 1994] in 
that it genuinely can predict parallel elements. Although 
both approaches rely on a sorts/types hierarchy and on 
some kind of overwriting to carry out this task, our ap- 
proach readily exten ds t o gapping cases whe reas as we 
shall see (cf. section 3.2) [ [Grover et al., 1994 1 doesn't. 

We proceed as follows. First we present a sort-based 
abductive calculus for parallelism and show that it pre- 
dicts parallel elements. We then show how this abduc- 
tive calculus can be combined with HOU thus yielding an 
integrated treatment of parallelism and discourse seman- 
tics. We then conclude with pointers to further research 
and related work. 



2 Defining discourse parallelism 



In li nguistic t heories on discourse c oherence |Kehlcr 
1995i , ellipsis JPalrymple et al, 199l[ (henceforth DSP; 

1996 [, the notion of par- 
In particular, the HOU- 



and corrections [ [Gardent et a 
allelism plays a central role, 
based approaches presuppose a theory of parallelism 
which precomputes the parallel elements of a pair of ut- 
terances. For instance, given the utterance pair Jon likes 
golf. Peter does too, DSP's analysis of ellipsis presup- 
poses that Jon and Peter have been recognised as being 
parallel to each other. 

Similarly, discourse theories of parallelism also assume 
parallelism st ructuration. According to [ Hobbs, 199C| ; 
Kehler, 1995| for instance, there is a class of discourse 



relations (the resemblance relations) which involve the 
inferring of structurally parallel propositions and where 
arguments and predicates stand in one of the following 
configurations: 
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where a,b represent argument sequences; ai,bi are any 
elements of these sequences; and S- and T-Ent are the 
propositions entailed by the two (source and target) par- 
allel utterances. Furthermore, entities are taken to be 
similar if they share some reasonably specific property 
and contrastive if they have both a shared and a com- 
plementary property. 

Again, the parallel elements {ai and bi) are taken as 
given that is, the way in which they are recognised is not 
specified. In what follows, we present a computational 
theory of parallelism which predicts these parallel ele- 
ments. The model is a simple abductive calculus which 
captures Hobbs and Kehler's notions of parallelism and 
constrast as they are given above. We make the sim- 
plifying assumptions that contrast and parallelism are 
one and the same notion (we speak of contrastive or c- 
parallelism) and that the properties p used in determin- 
ing them are restricted to sorts from a given, domain- 
spec ific sort hierarch y. Thus we can use sorted type the- 
ory I Kohlhase, 1994 1 to model similarity and contrastive 
parallelism. 

2.1 Sorted Logic 

Sorts correspond to the basic cognitive concepts. Logi- 
cally they can either be seen as unary predicates or as 
refinements of the types. The intuition behind this is 
that the universe of objects of a type a is subdivided in 
subsets which are represented by sorts A,B, . . .. Since 
these can in turn be subdivided into subsets, the sorts 
are ordered by a partial ordering relation < in a so-called 
sort hierarchy^ 

Just as in the case of types, every formula has a sort, 
that can be computed from the sorts of the constants and 
variables occurring in it. In fact, formulae can have mul- 
tiple sorts, corresponding to the fact that the intersection 
of the sets represented by their sorts can be non-empty. 

For this paper we assume a fixed finite set of sorts for 
each type. For the base type e, we will use the following 
sort hierarchy in our examples. 

Note that the intersection of the sorts Male and DOG 
is non-empty, since the constant Spot has both sorts. 
If we want to make this explicit, we can give Spot the 
intersection sort Male&Dog. Even though we as- 
sume the simple sorts (i.e. the non-intersection ones) 
to be non-empty, the intersection sorts can in general 



^For the purposes of this paper, we assume the sort hi- 
erarchy to be given. For applications, hierarchies could be 
generated from domain representations in KL-ONE like for- 
malisms commonly used in NL systems. 
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Figure 1: Sort hierarchy of type e 

be. For instance, the sorts Animate and Inanimate 
are disjoint, since they are complementary. The exis- 
tence of complementary sorts allows us to model the 
requirements for parallel elements quite naturally. Two 
formulae A and B (of any type) are similar if they have 
a common sort; they are contrastive, if they have a dis- 
tinguisiiing sort D, i.e. if A has sort E) but B has sort 
-iB or vice versa and finally they are c-parallel, iff they 
are both. 

For instance Jon and Mary are parallel, since both are 
of sort Human, but Jon has sort Man, whereas Mary 
has sort Woman = -iMan&Human&Female < -iMan 
and therefore Mary also has the distinguishing sort 
-iMan. This supports DSP's analysis of 

Jon likes golf, and Mary likes golf. 

For the higher-type, the sort hierarchies of lower type 
induce further sorts: For any sorts A and B of types 
a and /3, A B is a functional sort of type a — > 
(3. We call sorts that do not contain an arrow basic 
sorts. Similarly, the sort hierarchy of lower type induces 
subsort relations: B ^ C < A D, is entailed by A < B 
and C < D. Furthermore, the resulting sorts can be 
further subdivided by functional base sorts, i.e. sorts 
that do not contain an arrow, but are of functional type. 

For our examples we will use the following sort hier- 
archy of type e ^ e ^ t 
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Figure 2: Sort hierarchy of type e ^ e ^ t 

2.2 Computation of parallelism 

Given the above analysis, the relations support and 
oppose are c-parallel, since they have both a common 



sort (Social) and a distinguishing sort (Friendly). 
Further, in 

Jon supported Clinton, but Mary opposed him. 

paraUehsm theory should predict that the full first ut- 
terance Jon supported Clinton is c-parallel to the second 
namely, Mary opposed him. However, the sort t does 
not have subsorts that license this. Rather than divid- 
ing t into cognitively unplausible sorts, we propose an 
abductive equality calculus that generates all possible 
explanations, why a pair of formulae could be c-parallel, 
based on the respective sort hierarchies. The calculus 
manipulates two equalities =" for similarity and =p for 
c-parallelism. The inference rules given in figure ^ give 
the derivation from figure ^ that explains the parallelism 
in terms of assumed contrastivity and similarity of the 
components. We have put the justifications of the ab- 
ducibles in boxes. Note that this calculus gives us the 
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Figure 3: The abductive Calculus for Parallelism 

explanation that Jon supported Clinton is c-parallel to 
Mary opposed him, since Jon is c-parallel to Mary and 
support is c-parallel to oppose and finally, we can make 
Clinton and him similar by binding him to Clinton. 

Of course, there is a similar derivation that makes 
Mary and Jon similar and finally one that makes support 
and oppose similar but Mary and Jon c-parallel. Thus 
we have the problem to decide which of the different sets 
of abducibles is the most plausible. 

For this it is necessary to give a measure function for 
sets of abducibles. For instance the three pairs 

Jon =^ Peter Jon Spot Jon n 

are obviously ordered by increasing plausibility. We ob- 
serve that this plausibility coincides with the distance 
(the length of the connecting path) from the least sorts 
of the objects to common sort. Therefore, our approach 
is to derive plausibility values for abducibles from the 
justifications of abducibles by calculating distances in 
the sort hierarchies. 
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Figure 4: Jon supported Clinton, but Mary opposed him. 
3 HOU with Parallelism 

In recent approac hes to ellipsis l^alrymple et al, 199l| | 
and deaccenting | ]G ardent et ai, 1996| ; pardent, 1997 f, 
both parallelism and higher-order unification are central 
to the semantic analysis. For instance DSP analyze a 
VP-ellipsis such as 

Jon likes golf, and Mary does too. 

as being represented by l{j,g) A R{m) where R repre- 
sents the ellipsis does, whose semantic value is then de- 
termined by solving the equation R{j) = l{j,g). The 
motivation for having j occurr in the left-hand side of 
the equation is that j represents a c-parallcl element. 
This is where parallelism and the assumption of paral- 
lelism structuration come in. On the other hand, Higher- 
Order Unification is also essential in that it is used to 
solve the equation and furthermore, it is shown to be a 
crucial ingredient in attaining wide empirical coverage 
(in particular, it is shown to successfully account for the 
interaction of ellipsis with quantification, anaphora and 
parallelism) . 

However, it is also known that a pure form of HOU 
is too powerful for natural language and that a more 
restricted version of it namely, Higher-Order Coloured 
Unification (HOCU) is more adequate in that it helps 
prevent over-generation i.e. the prediction of linguis - 
tically invalid readings [ Gardent and Kohlhase, 1996 |. 
To see this, consider again the example just discussed. 
Given the stipulated equation, HOU yields two values 
for R namely, XX.l{X,g) and XX.l{j,g), of which only 
the first value is linguistically valid. To remedy this, 
DSP postulate a Primary Occurrence Restriction 
(POR): the term occurrence representing the element 
which is parallel to the subject of the elliptical utter- 
ance, is a primary occurrence and any solution contain- 
ing a primary occurrence is discarded as linguistically 
invalid. For instance, j is a primary occurrence in the 
equation l{j,g) =^ R(.j), so the solution R = XX.l{j, g) is 
invalid. [[Gardent and Kohlhase, 1996[ show that DSP's 



POR can be expressed within HOCU because it uses a 
variant of the simply typed A-calculus where symbol oc- 
currences can be annotated with so-called colours and 
substitutions must obey the following constraint: 

For any colour constant c and any c-coloured 
variable Vr, a well-formed coloured substitu- 



tion must assign to Vc a c-monochrome term 
i.e., a term whose symbols are c-coloured. 

In this setting the POR can be expressed by coloring 
the primary occurrence j with a colour pe but R with a 
colour -ipe. Due to the constraint above, this in effect, 
enforces the POR. 



More generally, [Gardcnt and Kohlhase, 1996 1 argue 



that HOCU rather than HOU, should be used for se- 
mantic construction as it allows a natural modelling of 
the interface between semantic construction and other 
linguistic modules. In what follows, we therefore as- 
sume HOCU as the basic formalism and show how it can 
be combined with the abductive calculus for parallelism, 
thereby providing an integrated framework in which to 
handle parallelism, ellipsis and their interaction. 

3.1 Abductive Reconstruction of 
Parallelism 

As just mentioned, we need a basic inference procedure 
that is a mixture of higher-order colored unification and 
the sorted parallelism calculus introduced above. The 
problem at hand is to make colored sorted formulae sim- 
ilar or c-parallel. For an algorithm ARP we build up 
on a sorted version of HOCU (which can be obtained 
by a straightforward combination of color techniques 
from llHutter and Kohlhase, 1995| with sorted meth- 



ods from I Kohlhase, 1994| ) but instead of simply having 



equations for sorted /3ry-equality, we also add the equa- 
tions for c-parallelism and similarity to the unification 
problem as special equations and 

The ARP calculates sufficient conditions for a given 
set of input equations by transforming systems of equa- 
tions to a normal form from which these can be read 
off. 

Decomposition rules consist in those from figure |^ with 
the difference that the rule for abstractions transforms 
equations of the form Xx.A =* Xy.B to [c/xJA [c/y]B, 
and Xx.A =* B to [c/x]A Be where c is a new con- 
stant, which may not appear in any solution. Further- 
more, there is a rule for colored constants that decom- 
poses an equation Ca Cb into the color equation a = b. 

The variable elimination process for colour variables 
is very simple, it allows to transform a set f U {A = d} 
of equations to [d/A]£ U {A = d}, making the equation 
{A = d} solved in the result. For the formula case, elim- 
ination is not that simple, since we have to ensure that 
|(T(a;A)| = |cr(a::B)| to obtain a C-substitution a. Thus 
we cannot simply transform a set £ U {x^ = M} into 
[M/xd]£ U {xd = M}, since this would (incorrectly) 
solve the equations {a;c — fc,Xd — gd}- The correct 
variable elimination rule transforms £ U {xd — M} into 
a{£) U {xd — M, — M^, . . . , x^^ = M"}, where are 
all colours of the variable x occurring in M and £, the 
M' are appropriately coloured variants (same colour era- 
sure) of M , and a is the C-substitution that eliminates 
all occurrences of x from £. 



Due to the presence of function variables, systematic 
application of these rules can terminate with equations 
of the form Xc{s^ ^ ■ ■ ■ , s") — hd{t^, . . . ,t"^)- Such equa- 
tions can neither be further decomposed, since this would 
loose unifiers (if G and F are variables, then Ga = Fb 
as a solution Xx.c for F and G, but {F = G,a = b} is 
unsolvable), nor can the right hand side be substituted 
for X as in a variable elimination rule, since the sorts 
would clash. The sorted, colored variant of Huet's clas- 
sical solution to this problem is to instantiate Xc with a 
c-monochrome formula that has the right sort B„ A 
(that of Xc) and the right head hd (which we assume to 
have sort 7^ A). These so-called general bindings 
have the following form: 



where the are new variables of sort B„ — > 7^ and 
the are either distinct colour variables (if c G CV) or 
ei = d = c (if c e C). If /i is one of the bound variables 
then Q'^ is called an imitation binding, and else, {h 
is a constant or a free variable) , a projection binding. 

The general rule for flex/rigid equations transforms 
{xc(s\...,s") = hd{t\...,t"')} into {xc{s\...,s^) = 
hd{t^ , ■ ■ ■ , t"^) , Xc — G'c}, which in essence only fixes a 
particular binding for the head variable Xc- It turns ou t 
(for details and proofs see [ Hutter and Kohlhase, 1995 1) 
that these general bindings suffice to solve all fiex/rigid 
situations, possibly at the cost of creating new fiex/rigid 
situations after elimination of the variable Xc and decom- 
position of the changed equations (the elimination of x 
changes Xc{s^, ■ ■ ■ , s") to Gci^^^ ■ • ■ i *") which has head 
h). This solution for pure equations has to be adapted 
to the more general similarity and contrastivity relations 
and where we have to provide further imitation 
rules. In particular, for an equation X^JJ =p KV we 
have to allow imitation bindings t/^ ^ for for any 
constant k that is contrastive to h and analogously for 



3.2 Gapping and ARP 

We now illustrate the workings of ARP by the following 
example 

Jon likes golf, and Mary too. 

where the second clause is a gapping clause in that both 
the verb and a complement are missing. This example 
clearly illustrates the interaction of parallelism with se- 
mantic interpretation: if the parallel elements are Jon 
and Mary, the interpretation of the gapping clause is 
Mary likes golf, but if conversely Mary is parallel to golf 
then the resulting interpretation is Jon likes Mary. Al- 
though the first reading is clearly the default, the second 
can also be obtained - in a joke context for instance. In 
what follows, we show that ARP predicts both the am- 
biguity and the difference in acceptability between the 
two possible readings. Additionally, we show that DSP's 



a-priori labelling of occurrences as primary or not pri- 
mary can now be reduced to a more plausible constraint 
namely, the constraint that Mary is a parallel element 
which has exactly one parallel counterpart in the source 
(or antecedent) clause. 

The analysis is as follows. First, we follow DSP and 
assign the above example the representation 

where R stands for the missing semantics. However, in 
contrast to DSP, we do not presuppose any knowledge 
about parallelism in the source utterance and determine 
the meaning of R from the equation 



which can be decomposed to the equations 



WOMAN- 



*(mpe 



which only says that the propositions expressed by Jon 
likes Mary and golf too stand in a c-parallel relation^. 
The rationale for the colors in this equation is that Mary 
must be a parallel element in the target utterance. For 
Jon and golf in the source utterance, we do not know 
yet which of them will be a parallel element, but it can 
be at most one of them, which we code by giving them 
unspecified but contradictory color^ Finally, R gets the 
color -ipe, since it may not be instantiated with formulae 
that contain primary material (POR). 

Since the elided material in gapping constructions and 
VPE may only copy material from the source utterance 
(and may not introduce new material) we add the con- 
straint to ARP that =P and imitations may only be 
applied to equations, where the head is pe-colored. We 
call this the copying constraint for gapping and VPE. 
It ensures that whenever two elements are similar but 
not identical, then they must be primary, since they are 
parallel. 

Let us now go through the ARP computation to see 
that our analysis obtains exactly the desired readings 
and to gain an insight of the mechanisms employed 
therein. 



The iulLial uquaLiuu is a flex/iigid paii, wheit; uuly Lhu 
stri ct imitation^ rule is applicable (there is no projection 
binding of sort WOMAN t). So, we obtain the binding 
XZ.l{H^peZ){K^peZ), where H and K are new variables 
of sort Woman Human. Eliminating this equation 
yields the equation 



l{jA,g-,/\) l{H^pempe){K^p, 



m 



^By contrast, an extension of DSP's analysis to gapping 
would posit the equations l{j,g) = R{j) and l{j,g) = Rig) 
thereby postulating both the parallel elements, and the am- 
biguity of the gapping clause. 

^Clearly, this coding is not general enough for the gen- 
eral case, where there are more than one parallel elements in 
the target utterance, we leave a general treatment to further 
work. 

*Note the copying constraint is at work here. 
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For the variable H^pe in the first equation both the imi- 
tation binding XZ.j^pg. and the projection binding XZ.Z 
are possible. 

In the first case, we have the equation ja =^ j^pe, 
which entails that A = -ipe leaving us with the second 
equation (we can eliminate double negations on colors) 



K m —P 

*- — ipe " *'pe 



5pe 



Again we have the possibility of imitate or project. Since 
the imitation binding XZ.gp^ for Kp^ leads to a color clash 
in (7pe =P g^pe, only the projection binding XZ.Z yields a 
solution, since the resulting equation mpe =p gpe is valid, 
since golf and mary share the sort Real. 

If, on the other hand, we choose the projection binding 
for -ffpe, then variable elimination yields the equation 



'pe; 



which is valid, since Jon and Mary share the 
sort Human and which entails that A = pe leaving us 
with the second equation 



K- 



5-pe 



Again we have the possibility of imitate or project. This 
time, the imitation binding XZ.g^p^ for K^p^ leads to the 
trivialy valid equation g^pe =p g^pe, while the projection 
binding XZ.Z yields the equation TOpe g^pe, which 
must be unsolvable, since the colors clash. 

If we collect the bindings, we arrive at the two solu- 
tions XZ.l{Z,g) and XZ.l(j, Z), which correspond to the 
readings Mary likes golf and Jon likes Mary. Note that 
since the similarity of Jon and Mary is stronger than 
that between Mary and golf the first reading is pre- 
ferred, while the second reading may only be obtained in 
the context of a joke. Note also that the use of colours 
(i.e. the constraint that Mary has exactly one parallel 
counterpart in the source) correctly rules out the math- 
ematically valid solution XZ.l{Z,Z) where Mary would 
be analysed as contrasting with both Jon and golf. 



We now briefly compare our approach with | Grovcr ei 



ai, 1994 1 where an account of VP-cUipsis is given which 



uses first-order default unification on feature-structure 
semantic representations and claims to predict the par- 
allel elements. The underlying idea is that parallel ele- 
ments have identical thematic roles. Although the ap- 
proach works well for VP-ellipsis, it is only because in the 
VPE case, the parallel element of the elliptical clause is 
known to fill the agentive role. However in gapping cases, 
part of the resolution problem is to determine the the- 
matic role of the constituents appearing in the elliptical 
clause. For instance, in Jon likes golf and Mary too, the 
thematic role of Mary is either agent or patient. Unfor- 
tunately, the only obvious way to incorporate this into 
a default unification approach would be to augment it 
with variables over features - an essentially higher-order 
construct. 



3.3 Controlling ARP 

Clearly, a naive implementation of the ARP calculus as 
sketched above will be intractable, since the set of ab- 
ducibles is much too large. However, most abducibles are 
very implausible and should not be considered at all. As 
in all implementations of abductive processes, the search 
for abducibles has to controlled, which in turn calls for a 
quality measure of abduced equations. A standard (but 
not very imaginative^) measure would be the conceptual 
distance of the sorts justifying the equation, (i.e. the 
number of subsorts crossed to reach the common and 
discerning sorts). In our example, the rating of m —P g 
is 6, while that of m =p j is 2, justifying the claim that 
the reading Mary likes golf, is more plausible than Jon 
likes Mary. Since all other readings are either ruled out 
by the colors or are even more implausible, e.g. an A* 
implementation of ARP will only derive these, iff given 
an appropriate threshold. Since the aim of this paper is 
to establish the principles of parallelism reconstruction, 
we will not pursue this here. 

4 Conclusion 

We have given a sketch of how to develop a computa- 
tional framework for calculating parallelism in discourse. 
This approach is based on the HOCU variant of DSP's 
HOU account of ellipsis, but unlike that approach does 
not presuppose knowledge about the parallel elements. 
Instead, it computes them in the analysis. 

Parallelism can be seen as affecting the interpretation 
of the second of two parallel utterances in mainly two 
ways: it can either constrain an anaphor to resolve to its 
source parallel counterpart (this is the case for instance, 
in the gapping example discussed above); or it can add 
to its truth conditional content. For instance, in 

Jon campaigned hard for Clinton in 1992. 
Young aspiring politicians often support their 
party's presidential candidate 

parallelism enforces a reading such that Jon is under- 
stood to be a young aspiring politician and Clinton is 
understood to be Jon's party's presidential candidate. 

In future work, we plan to investigate these two as- 
pects in more details. As for the interaction of paral- 
lelism with binding, one important question is whether 
our proposal preserves DSP's insights on the interaction 
of parallelism with ellipsis, anaphora and quantification. 
On the other hand, to account for the incrementing effect 
of parallelism on semantic interpretation, the proposal 
will have to cover the discourse relations of exemplifica- 
tion and generalisation. Note however that the proposed 
interleaving between HOU and abductive calculus gives 
us a handle on that problem: mismatches between se- 
mantic structures can be handled by having the calculus 

^Clearly, a more sophisticated measure would include con- 
cepts like the specificity of the solution. 



be extended to abstract away irrelevant structural differ- 
ences (this would account for instance for the fact that 
in our example, a temporal modifier occurs in the source 
but not in the target) whereas sorted HOU can be used 
to infer information from the most specific common sort 
(in this case, the sort of young aspiring politicians). 
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